Image rotation correction for video or photographic equipment

ABSTRACT

Image recording apparatus such as a camera ( 2 ) stores data ( 8 ) representing still or moving image data on a data carrier ( 12 ). A rotation sensor ( 14 ) measures the angle of orientation of the camera The stored data is based both on the image data and the output angle data to allow the captured image to be rotated for display to have the true horizontal displayed as horizontal.

The invention relates to the processing of image information,particularly in video or photographic equipment, and to apparatus forrecording images such as video or photographic images.

Cameras, such as video or still digital cameras, are frequently used byamateurs who may have difficulties in correctly orienting the camera.There is thus a need for a camera capable of reducing the effect of poororientation.

U.S. Pat. No. 5,764,291 describes a camera having an orientation signalthat outputs one of four values, indicating left, right, inverted orupright operation. This enables the camera orientation to be correctlyrecorded as one of these four orientations.

According to the invention, there is provided an image recordingapparatus comprising: a lens; a means for capturing an image projectedby the lens; a rotation sensor for providing an output representing theangle of the captured image to the horizontal to an accuracy of 5° orbetter; and a data carrier for recording data based on both image datarepresenting the captured image and angle data representing the anglemeasured by the rotation sensor so that a rotated image corrected tohave the true horizontal in the captured image displayed as horizontalcan be displayed from the data recorded on the data carrier.

In this way, the image data can be used to output a corrected image withthe horizontal of the scene captured by the camera displayedhorizontally even if the camera is not exactly aligned.

The image recording apparatus according to the invention thus has adifferent effect and purpose to that disclosed in U.S. Pat. No.5,764,591. U.S. Pat. No. 5,764,591 discloses recording one of four basiccamera orientations, i.e. portrait and landscape each in two possibleorientations. In contrast, the apparatus according to the inventioncorrects for fine misalignment between the actual camera orientation andthe desired basic orientation.

Preferably, the accuracy of the rotation sensor is much better than 5°,i.e. 2° or better, or further preferably 1° or better, or even 0.5° orbetter.

Conveniently, the stored image data can simply be the data output by thedigital recording sensor and the angle measured by the rotation sensor,leaving correction to occur on playback.

Alternatively, the camera may include a correction module for correctingthe image data captured by the camera based on the output of the digitalrecording sensor and storing the corrected image data in the memory.

The camera may be a stills camera or a video camera. The data carriermay be any convenient form of memory, including a tape, a flash memory,or any other memory suitable for storing digital data.

In a particularly beneficial embodiment, the digital recording sensorhas a plurality of pixels arranged as a normal rectangular frametogether with additional pixels arranged around the periphery of thenormal frame. The additional pixels are not used for an image that isrecorded with the camera vertical, but provide the additional pixelsneeded to deliver a rotated image with the same size of rectangularframe.

Alternatively, the rotated image may be resized slightly compared withthe normal image.

In a particularly preferred embodiment, the pixels of the digitalrecording sensor are arranged as a circular array to allow rotation ofthe image to an arbitrary angle without needing to resize the imageframe.

The pixels may be arranged as a hexagonal array.

A plurality of sensors may be provided to allow for image correction ina number of planes, for example corrections for rotation and tilt. Thecamera may be able to use the information from the sensors to record apanoramic image by stitching together a plurality of individual images.

The invention also relates to a camera as set out above incorporatedinto a further device, including for example a mobile telephone, orcomputer.

In another aspect, the invention relates to a method of processing imageinformation, including capturing image information in a camera;measuring the orientation of the camera with respect to vertical to anaccuracy of better than 5°, and recording image data based on the imageinformation and the measured orientation of the camera.

The step of recording image data may include recording the imageinformation and the measured orientation of the camera as separatepieces of data. The method may further include correcting the recordedimage data based on the measured orientation of the camera; andreplaying the corrected image information.

For a better understanding of the invention, embodiments will now bedescribed, purely by way of example, with reference to the accompanyingdrawings, in which:

FIG. 1 is a schematic drawing of a camera according to the invention;

FIG. 2 is a schematic drawing of replay apparatus according to theinvention;

FIG. 3 is a schematic illustrating the operation of a method accordingto the invention;

FIG. 4 is an illustration of the frame area;

FIG. 5 illustrates a circular image frame;

FIG. 6 illustrates a frame aspect ratio of 760:480 in a circular imageframe; and

FIG. 7 illustrates a frame aspect ratio of 1:1 in a circular imageframe.

A camera 2 includes a lens 4 and a charge-coupled device (CCD) 6 whichacts as a sensor to capture image data 8 representing a scene 10 infront of the camera. The camera may record either still images or movingimages, in the latter case by capturing a sequence of items of imagedata 8 representing sequential frames of the moving image.

The camera 2 also includes memory 12 for recording the image data 8. Thememory is in the example shown a tape, although the skilled person willbe familiar with many other forms of memory suitable for storing stillor video images and these may also be used. For example, the memory maybe a solid state memory such as a flash memory, a removable memory card,a magnetic disc, an optically recordable disc or film.

A rotation sensor 14 is provided that measures the angle of the camerato the vertical and outputs a corresponding measurement signal. Thisangle is recorded in the memory 12. The rotation sensor is an accurateaccelerometer.

Suitable solid state accelerometers are commercially available atmoderate cost with an accuracy of 1°. These accelerometers include amicro-etched tongue that bends under gravity and so changes acapacitance. Improved performance is available at higher cost. Othersuitable devices are sold as “tilt meters”.

The camera 2 is under the control of camera processor 16. The cameraprocessor 16 may be arranged to be switchable between a correction modemeasuring and storing the angle information for correcting the image anda normal mode in which the rotation sensor is not used.

A playback device 20 is provided that can accept memory 12 and processthe recorded image data 8 and rotation angle in replay processor 22. Theprocessed image data is played back on display device 24, for example atelevision set. It will be appreciated by the skilled person that thereplay processor 22 may be incorporated in camera 2 and may indeed bethe same processor as camera processor 16.

The processor 22 can operate in a correction mode that corrects forinaccurate orientation of the camera with respect to the vertical. Thus,the processor 22 uses the recorded rotation angle information to causean image to be displayed on display 24 that has the horizontalhorizontal, even if the camera was slightly misaligned when the imagewas recorded.

FIG. 3 illustrates the use of the system. FIG. 3 a shows the image thatthe user should have taken i.e. with the camera horizontally aligned.FIG. 3 b shows the image taken with a camera angle at an angle of about10° to the vertical. The true horizontal in the image is thus also at anangle of 10° to the horizontal in the recorded image.

The image as shown in FIG. 3 b is recorded in the memory together withthe data item that records the camera angle.

On replay, the processor 22 calculates the image on display 24 as shownin FIG. 3 c, with a rotation correction applied using the camera itemdata angle by replay processor 22 so that true horizontals appear ashorizontal. Some of the image frame of the rotation corrected image isblank, as may be seen. Accordingly, the image is cropped to eliminateblank space as illustrated in FIG. 3 d. Frame 40 represents the recordedimage and frame 42 represents the image frame of the displayed image.

As illustrated in FIG. 3 e, the corrected image 42 is resized so thatthe final image frame 44 is the same size as original image frame 40.

In a modification of the invention, illustrated in FIG. 4, the recordedimage area 50 is greater than that of the nominal image frame 40 inorder that the image may be rotated without resizing. Marginal pixels 52outside the nominal frame 40 may be switched on only when required toavoid excessive power drain when the automatic alignment feature is notin use.

In a further modification, the sensor array may be circular to enablearbitrary rotation to take place without any loss at the edges. FIG. 5illustrates such a circular sensor 60 with nominal image frame 40.Rotation to any angle is possible without size reduction. The individualpixels of circular sensor 60 may be arranged on a hexagonal grid.

The aspect ratio of image frame 40 in circular sensor array 60 changesthe ratio of the circle within the image frame. For example, a rectangleof aspect ratio 760:480 as illustrated in FIG. 6 gives a useful arearatio of 57.48% of the area of the circle, whereas a square of aspectratio 1:1 as illustrated in FIG. 7 gives a useful aspect ratio of63.66%. The maximum area of the circular sensor is used in asubstantially square frame, of aspect ratio in the range 1:0.7 to 1:1.4,preferably 1:0.9 to 1:1.1.

The rotation sensor 14 may be a multiple angle rotation sensor, capableof measuring orientation about more than 1 axis and storing the resultin memory 12. The sensor may include a plurality of accelerometers. Thestored information may then be used, for example, to stitch together ahigh quality image from several successive image to make up a panoramicimages using the orientation information to accurately align and stitchtogether the different still images.

In an alternative embodiment, the processor 16 may function as acorrection module by taking the image captured by the sensor 6 and theangle of orientation of the camera 2 as output by rotation sensor 14,correcting the image data so that the image is accurately horizontal andstoring the corrected image in memory 12.

The invention is not limited to the embodiments described above and theskilled person will be aware of many modifications that may beimplemented.

In particular, the invention is applicable to any form of camera,whether a stills camera, a video camera, or a camera capable of bothstills and video operation.

Although the invention is described with relation to a digital camera,the invention may also be used with an analogue camera recording imagesdirectly on a recordable medium acting as the memory as well as therecording sensor. It is in this case necessary to also record the angleinformation on the recordable medium, but technology for recordingadditional information is well known to the skilled person in both thefields of motion picture photography and still picture photography.

The invention is also applicable to the case where the camera isincorporated in other apparatus, such as a mobile telephone or computer.

1. Image recording apparatus comprising: a lens; a means for capturingan image projected by the lens; a rotation sensor for providing outputangle data representing the angle of the captured image to thehorizontal to an accuracy of 5° or better; and a data carrier forrecording data based on both image data representing the captured imageand angle data representing the angle measured by the rotation sensor sothat a rotated image corrected to have the true horizontal in thecaptured image displayed as horizontal can be displayed from the datarecorded on the data carrier.
 2. Image recording apparatus according toclaim 1 wherein the rotation sensor has a precision of better than 1°.3. Image recording apparatus according to claim 1 or 2 wherein the meansfor capturing an image is an optical sensor for outputting a signalrepresenting the image.
 4. Image recording apparatus according to claim3 wherein the apparatus is arranged to store the data output by therecording sensor and a data item representing the angle measured by therotation sensor.
 5. Image recording apparatus according to claim 3further comprising a correction module for correcting the captured imagedata based on the output of the rotation sensor and storing thecorrected image data in the memory.
 6. Image recording apparatusaccording to any preceding claim wherein the digital recording sensorhas a plurality of pixels arranged as a normal rectangular framecorresponding to an unrotated image together with additional pixelsarranged around the periphery of the normal frame to provide additionalimage data for use when rotating the captured image.
 7. Image recordingapparatus according to any preceding claim wherein the pixels of thedigital recording sensor are arranged as a circular array to allowrotation of the image to an arbitrary angle without needing to resizethe image frame.
 8. Image recording apparatus according to any precedingclaim comprising a plurality of sensors to allow for image correction ina number of planes.
 9. Image recording apparatus comprising: a lens; asensor for capturing an image projected by the lens and outputting imagedata; a rotation sensor for measuring the angle of rotation of the imagerecording apparatus about a horizontal axis and outputting correspondingangle data; and a data storage device for storing data both the imagedata and the angle data.
 10. A method of processing image information,including: capturing image information in a camera; measuring the angleof orientation of the captured image information with respect to thetrue horizontal to an accuracy of better than 5°, and recording imagedata based on the image information and the measured orientation of thecamera.
 11. A method according to claim 10, wherein the step ofrecording image data records the image information and the measuredorientation of the camera, the method further comprising the steps of:correcting the recorded image data based on the measured orientation ofthe camera; and replaying the corrected image information.